1. Technical Field
Several aspects of the present invention relate to an image sensor that is allowed to expose photoelectric conversion elements to light on a line-by-line basis. In particular, the invention relates to an image sensor and an image taking apparatus that are allowed to take an image at a frame rate corresponding to a normal exposure time and simultaneously take an image at a frame rate higher than the normal frame rate, and a state inspection system including the image taking apparatus.
2. Related Art
Due to problems such as increases in labor cost or from a humanitarian viewpoint of reductions in long-time simple work, a camera has been used in a manufacturing process or an inspection process. For example, by performing inspection using a camera or sorting out assembly parts using a camera, the shape of the subject is instantly measured and recognized. Simultaneously, the color or the like of the subject is also measured. For example, the shapes and colors of agricultural products, processed products thereof or the like are simultaneously inspected and these products are sorted based on the inspection result. Among inspection apparatuses using such a camera are agricultural product visual inspection apparatuses disclosed in JP-A-2005-62008 and JP-A-09-318547. These apparatuses measure both the shapes and colors of agricultural products using a charge coupled camera (CCD) camera and perform determination processes based on the measured shapes and colors. It is desired that these inspection apparatuses speed up for higher efficiency and the costs be reduced.
A CCD camera is typically used as an industrial camera for use in inspection apparatuses as described above. On the other hand, complementary metal oxide semiconductor (CMOS) cameras are also in widespread use mainly for entertainment and photography purposes. While CMOS cameras have become cheaper than CCD cameras and the image quality thereof has been improved, CCD cameras are still in the mainstream. This is partly because CCD cameras have a “global electronic shutter” function.
A CMOS sensor (CMOS image sensing element) typically performs exposure and reads out signals using a rolling shutter method in terms of the structure thereof. In the rolling shutter method, signals are read out from the lines (pixel lines) of the light reception area of the image sensor at different timings. Therefore, when an image of a moving subject is taken, the taken image is distorted. This phenomenon is remarkable when the subject is moving fast.
FIG. 6B is a drawing showing the timing of exposure of each line and the timing of readout of a pixel signal from each line in the related-art rolling shutter method. In FIG. 6A, the vertical axis represents the numbers of the lines included in the image sensor and the horizontal axis represents the time. For the sake of clarity, assume that there are eight lines, L1 to L8. In the rolling shutter method, while L1, L2, L3, . . . , L8 are sequentially scanned, readout and reset are performed on the scanned lines at timings indicated by arrows (↑) in FIG. 6B. Therefore, as shown in FIG. 6B, time lags occur among the exposure/readout timings of the lines. For example, a time lag corresponding to approximately one frame occurs between L1 and L8. When an image of a moving subject is taken, this time lag distorts the image. The distortion changes the shape of the subject in the image. This is a shortcoming in terms of shape recognition. A CCD camera also causes an image blur phenomenon when a subject moves. Accordingly, the shape recognition accuracy of the CCD camera also reduces depending on the moving speed or the like of the subject.
In order to take an image of a moving subject and recognize the shape of the subject while preventing occurrence of image distortion or image blurs, the frame rate must be increased. Unlike a CCD sensor, a CMOS sensor is allowed to increase the frame rate because a circuit can be mounted on the image sensor, the wiring is a high degree of freedom, and the like. For example, an image sensor having a high frame rate such as 180 fps has been developed for industrial application. In addition, there is a report that an image taken at a frame rate of 180 fps contains little waveform distortion. However, if the frame rate is increased, there occurs a problem that the exposure time is reduced, the output level of pixel signals is reduced, and thus the image quality (S/N) is reduced.
Hereafter, the amount of exposure will be described. A production facility or an inspection apparatus including an industrial camera adjusts the exposure amount by controlling both the exposure of the camera and the lighting environment, so as to improve the recognition accuracy. Specifically, the facility or apparatus optimally controls the aperture, the amount of illumination light, or the like so that an image of high quality is obtained at the frame rate of the camera (the rate corresponding to the normal exposure time is 30 fps), in other words, a sufficient signal level is outputted from an dark part of the subject and so that a bright part thereof is not saturated with signals (if there is a saturated part, the shape of the part will not be recognized). Therefore, such an industrial camera is significantly different from a camera for a photography purpose, which is expected to be used under various environments.
As a method for obtaining high image quality using a high frame rate (a frame rate N times higher than the normal frame rate), there are a method of making the illuminance of the lighting system N times higher and a method of making the pixel area of the image sensor N times larger for example. The former method is problematic in that it increases the running cost, that it is not environmentally friendly, and the like, while the latter method has a problem that it increases the cost of the camera.
Among methods for removing image distortion caused by a CMOS sensor is a method of providing a global shutter mechanism on an image sensor like in a CCD sensor. However, this method complicates the pixel structure, thereby disadvantageously reducing the pixel performance and increasing the cost of the image sensor.
Alternatively, there is a method of using two cameras and selectively using one of the two cameras as a high-speed camera for detecting the movement or shape of a subject and the other as a normal camera for detecting the color or texture of the subject, to obtain both high quality and high speed. However, this method has problems of high cost and high power consumption.